Additionally, we show that LM behaviors can be customized using different combinations of fine-grained reward models.

Language models (LMs) often exhibit undesirable text generation behaviors, including generating false, toxic, or irrelevant outputs. Reinforcement learning from human feedback (RLHF) - where human preference judgments on LM outputs are transformed into a learning signal - has recently shown promise in addressing these issues. However, such holistic feedback conveys limited information on long text outputs; it does not indicate which aspects of the outputs influenced user preference; e.g., which parts contain what type(s) of errors. In this paper, we use fine-grained human feedback (e.g., which sentence is false, which sub-sentence is irrelevant) as an explicit training signal. We introduce Fine-Grained RLHF, a framework that enables training and learning from reward functions that are fine-grained in two respects: (1) density, providing a reward after every segment (e.g., a sentence) is generated; and (2) incorporating multiple reward models associated with different feedback types (e.g., factual incorrectness, irrelevance, and information incompleteness). We conduct experiments on detoxification and long-form question answering to illustrate how learning with such reward functions leads to improved performance, supported by both automatic and human evaluation. Additionally, we show that LM behaviors can be customized using different combinations of fine-grained reward models. We release all data, collected human feedback, and codes at https://FineGrainedRLHF.github.io.

The digital world has been entirely transformed with the help of technological breakthroughs, and IoT (Internet of Things) is to be credited among AI (Artificial Intelligence), ML (Machine Learning), Data Science, and more. Internet of Things has been the futuristic concept of connecting and controlling our devices and items remotely. This future idea alone has brought drastic change within many industries that have seen improved processes, increased productivity, and many other benefits. However, one of the most significant contributions that IoT has made in assisting users with disabilities. We'll get to that thought in this article.

Amsterdam and Helsinki today became the first cities in the world to launch open AI registers that track how algorithms are being used in the municipalities. In a press release, the cities said the registers would help ensure that the AI used in public services operates on the same principles of responsibility, transparency, and security as other local government activities. "Algorithms play an increasingly important role in our lives," said Touria Meliani, Deputy Mayor of Amsterdam. "Together with the city of Helsinki, we are on a mission to create as much understanding about algorithms as possible and be transparent about the way we -- as cities -- use them. Today we take another important step with the launch of these algorithm registers."